Spatial Data Infrastructures – The Dynamic Generation

In the simulation domain, standards have supported interoperable solutions and the development of refined tools. Emerging standards and specifications in the spatial domain are likely to stimulate similar distributed processing systems. Spatial data infrastructures (SDI) allow the access to geographic information using standardized interfaces and web service technologies. Interoperability at the data level is achieved by defining the Geography Markup Language (GML) as an XML encoding for the transport and storage of geographic information, including both the spatial and non-spatial properties of geographic features. Following the publication of the WMS and WFS interface specifications by OGC, which describe the interchance and use of static geographic data, the expectation of access dynamic operations via the web is growing. The initial focus in OGC has been to investigate standardised interfaces for “Sensor Web Enablement” (SWE), in which the data source is a live sensor operating in near-real-time, rather than the conventional static data store. Note, however, that a simulator resembles a sensor regarding the provision of spatio-temporal data (it differs in the way how it estimates the value and in being temporally independent). In this article we discuss two approaches that allow the use of simulated data within the SDI world, as well as the integration of live data measured by real sensors and static data in simulation models. We show how decision makers in disaster management and in mineral exploration can make use of systems based on the SWE interfaces. Introduction Geographic information systems (GIS) have been a research topic within the spatial domain for years now. A lot of work has been conducted in regards to data integration and interoperability problems. Most conventional GIS use specific data models and underlying data storage strategies that do not interoperate among each other. Usually, making new data available to the system implies transformation processes that may cause loss of information, aside the tediousness and time consume these processes bring with them. The objective of Spatial Data Infrastructures is to overcome much of these problems. SDIs simplify the often labor intensive search for suitable data for a given task, allow direct access to the data and provide the required processing capacities ideally. In developing the concept and early deployment of SDI’s, the focus has been on standard information sources and catalogues of these. The primary role of geospatial web-service interfaces has been to provide a façade to static datastores managed by (usually) statutory data providers. However, there are many other potential sources of geospatial information. In particular, “live” sources, in form of sensors and other devices that may be polled for data in near-real-time. In many cases, of course, data from sensors is being continuously archived, and thus populates the static repositories in due course. In this case a sensor-access service may itself be merely a façade or convenience interface to a conventional source. But there are also precedents the other direction. In remote sensing potential data volumes are so large that it is only archived on-request. Or there may be temporary interest in a higher than normal acquisition rate, which would be enabled by having a sensor access service. To enable the latter use cases there is often a complementary requirement for services supporting dataacquisition scheduling. The complete information request may be quite elaborate, but will be composed of several pieces, concerning the tasking of the system followed by the retrieval of the result in the desired form. Furthermore, the complete transaction is distinctly asynchronous. An interesting variation on this model is where the information is generated computationally, by a simulation system. Most of the characteristics are shared with sensors: the information is generated ondemand only, the request will be highly parameterized, the system requires tasking and the response may be highly asynchronous, the results are often estimates of the description of some real phenomenon. However, simulations are usually distinguished by the results being time-shifted. The results of a simulation will usually be evaluated against the results of sensor systems. This is often as part of an inversion process, where the information of real interest is the constraints or initial conditions of the simulation. The Open GIS consortium (OGC) represents the most promising approach to establishing an integrated system of standardized spatial data infrastructures. It was founded in 1994 as a non profit organization and follows currently the mission “to deliver spatial interface specifications that are openly available for global use”(OGC 2002). It has more than 250 members, including public and private companies, universities, government agencies and other organizations from all over the world who are interested in building spatial interoperability. Thus OGC is one of the world leading organizations addressing interoperability issues within the spatial domain. The “Sensor Web Enablement (SWE)” initiative of the OpenGIS consortium has initially focused on standardized interfaces to access data provided by live sensors operating in near-real-time. The interfaces were designed to allow any arbitrary sensor type to described, and accessed. The first interface defined under the SWE initiative was the “Sensor Collection Service” (SCS) in which only the description of a sensor and the access to the provided data had been addressed. This was followed up with consideration of supporting the planning and tasking process of sensor measurements by suitable interfaces, in particular the “Sensor Planning Service” (SPS). The next generation of SWE will allow a much more dynamic use of SDIs, including the incorporation of simulation models. The principle is that sensors and simulators are both “procedures” that yield estimates of the description of a phenomenon, and in both cases typically involve the processing of “more primitive” values to produce results with greater semantic value. In this context, data provided within a spatial data infrastructure is no longer static and will become highly flexible due to the scenario building capabilities of the new service interfaces. Simulation Simulation is a key technology for describing, assessing, analyzing, forecasting, etc. the dynamics of real, planned or virtual processes or systems. These processes may operate in multiple dimensions of space and time. After a brief discussion of the interoperability aspect regarding simulation models, we will discuss first a web-based service architecture that integrates distributed heterogeneous GI and simulation services. Second, we will discuss an entirely standards based approach. In the following, the term “simulation” is understood in a rather broad sense. The use of simulation models within the World Wide Web is not a new idea. The web was used from the late nineties onwards to distribute individual components of simulation models. Initially, Java was seen as a sine qua non language, which led to the fact that existing simulation environments had been wrapped or ported to Java. . At the same time, CORBA, developed by the Object Management Group (OMG) was widely used as the architecture of first choice. Currently, the newest flagship of the OMG, the Model Driven Architecture (MDA), might be able to follow in the footsteps of the successful CORBA development. Although it is widely discussed in the simulation community, MDA shall not be discussed in this paper but will be investigated in future work. The next step of distribution was to create Java based management systems that allow a web based control over simulation components or models (Marr et al. 2000; Gebert and Osterburg 2002). For all that, the linkage of simulation models and GIS still remains file based data transfer in a non standardized way using similar exchange and reintegration mechanisms like GIS-to-GIS data exchange. OWS service chaining approach The Open GIS Consortium’s Web Services architecture provides a model for coupling geographic simulation and data systems that overcomes the aforementioned shortcoming of restricted data exchange. OWS approaches the deployment of distributed services at a coarse-grained level, based on passing XML encoded messages over http. The OWS interfaces define specific requests and responses that address a limited set of data models, such as geographic features, coverages, observations and sensors. The internal workings of the components are not prescribed. A data object may be delivered by a service by copying a file from cache, by polling a remotely deployed sensor array, or by computing the estimates using simulation software and then formatting it dynamically. The behavior of OWS is exemplified by the operations comprising the Web Feature Service (WFS) interface (Fig. 1). Figure 1 UML protocol diagram for WFS These are invoked as a series of messages, forming a dialogue between the client and service. The client first retrieves the capabilities of the service, which summarises the information available from this offering, primarily in terms of the feature types offered, and the geospatial area covered. The client then requests details of the format of the response for the feature types of interest. Finally, the client forms a suitable GetFeature request, and the server responds with an XML document in which the description of the selected objects is serialised. (Note that WFS may return other formats if available and requested.) Transactional operations (insert, update, delete) are also possible. Note that the basic implementation of this protocol, where the messages are carried over simple http, is an example of the REST architectural style (Fielding & Taylor, 2002). A complete system utilising web service interfaces will be composed of information servers, clients, and the middleware and registries necessary to broker connections between clients and servers. Many users will encounter OWS services through interactive desktop clients, including some capacity for visualisation, and a user interface through which conformant OWS requests are generated and routed to the appropriate server. However, client software will take many forms with various levels of client-side processing capability. Some software will act as both a client and server: brokering requests from a client and generating new requests on one or more primitive data sources, performing some processing tasks, and then returning the result back to the end-user. This operation mode is required for discovery middleware, but will also feature in coupled simulation systems. The idea is that a complete system can be assembled as a chain of webservices, with the output from one becoming input parameters to the next, under the control of marshalling or brokering services. Application Scenario: coupled simulations of mineralized systems Traditional mineral exploration has relied on the recognition of primary geophysical and geochemical signatures characterising the presence of ore-bodies. However, for many mineral occurrences, the signatures are indirect. The proposition is that, rather than focussing only on the direct phenomenon, efficient exploration should take into account the overall context within which minerals bodies form. This will allow exploration to be based on contextual evidence when primary anomalies are not present. But this requires that the connection between the overall system and ore-body is understood. Given the high number of degrees-of-freedom involved, simulation is an essential component of this strategy. Until now, efforts to incorporate geological simulations into the mineral exploration process have been characterised by ad-hoc use of observational constraints, cumbersome file-oriented data exchange and conversion, limited coupling between simulations dealing with different aspects of the complete physical process, and consequently a failure to provide information useful in exploration targeting in a timely manner. In collaboration with other statutory agencies and software vendors, CSIRO is developing a distributed geoscientific information, modelling and simulation system. The goal is to enable industry and government clients to efficiently evaluate potential and explore prospective locations (at all scales) for large high value ore bodies. To accomplish this, the system will use a complete set of earth process modelling and simulation tools which: • will provide solutions to most of the geological process-related problems related to targeting or evaluation of prospective areas • use the best sources of observational constraints (archived or live) • will provide solutions to complex problems in a time scale of no more than several weeks • will operate either individually or coupled together • can be accessed easily from a variety of distributed locations (e.g. explorationists’ computers). We describe CSIRO experience in implementing a distributed geoscience modelling and simulation system to support semi-automated scenario/hypothesis testing for interpretation of mining exploration data. The system operates using a network services model with remote users accessing the service running on CSIRO (and other) servers via either a thin-client web browser or network service aware desktop application. Simulations using this system are intended to reproduce the geological history of a region during periods of potential ore-body formation. The goal is to use predictions based in geological processes to support mineral exploration based on the entire geological context, rather than relying only on direct detection of primary geophysical and geochemical signatures. Our initial prototype was based on coupling four application software packages • 3-D geological model builder (Fig 2) • finite difference solver for high strain mechanical deformation and fluid flow (Fig 3) • finite element solver for chemistry • 3-D visualisation (Fig 4) Figure 2 Geological contacts (interfaces) modelled in a geological CAD package Figure 3 Zoned grid of different materials types used by numerical solver Figure 4 High strain zones where mineralisation is expected Interprocess communication used XML representations of the model state. The whole system is controlled through a browser hosted application (Figure 5). In the early version of the coupled geological simulation system many of the constraints (e.g. material properties) were entered manually. This is now being upgraded so that these values are accessed dynamically from standard databases. The database access will be modelled on WFS. Figure 5 data entry and service selection form for coupled geological simulation system We are now extending the system to incorporate additional components, such as • a library of problem-definition templates, to allow an end-user to rapidly design a suitable problem configuration through adjusting a minimum set of parameters • a genetic algorithm to automatically vary model parameters to search the parameter space for most probable scenarios • additional numerical solvers to handle a wider variety of modelling scenarios • more efficient mesh-generators • direct connections to sources of input data, such as material properties and field geometries, using WFS-like interfaces/. At the same time, the service architecture is being generalised, so that there is a routine pattern for the addition of further capabilities. Our focus is on the use of coarse-grained parallel computation where we are running the same simulation software with many different "inputs" to explore the parameter space. This is achieved using a simulation computational service concept where each Computational Engine (CE) is represented as a Web Service. Application Scenario Flood Management The flooding of the Elbe River and its tributaries in 2002 caused damages officially quoted as 9.2 billion Euros in Germany and additional 3 billion Euros for the Czech Republic. Figure 6 shows a fairly large lake in the right-hand side just south of the town of Wittenberg, where the Elbe River would just resemble a thin line normally. The image was taken August 20, 2002, by the Enhanced Thematic Mapper plus (ETM+), flying aboard the Landsat 7 satellite. When this image was taken, tens of thousands of people had been evacuated from their homes in Germany. Fifty thousand troops, border police, and technical assistance workers were called in to combat the floods along with 100,000 volunteers. Figure 6 Elbe River south of the town of Wittenberg (image by NASA) During the severe flooding, it was nearly impossible to get any valuable calculation about the development of the water masses. Simulation based scenario management services, allowing the calculation of multiple breaches or intentional openings in the dams would have simplified the decision making process: Where to evacuate people, where to open the dams to protect more valuable area from the water masses. Two different types of web based simulation services could help in these situations: Those that are continuously running and provide a calculation for certain moments in the future, and those simulations which have to be started individually after feeding with the mandatory parameters. The latter will allow scenario calculations and answer the question: What would be if...? The DALI-project (Distributed spAtio-temporaL Interoperable services) at the Institute for Geoinformatics addressed the issue of integrating simulation models into spatial data infrastructures by making use of the interoperability framework provided by OGC Sensor Web Enablement services (Simonis et al. 2003). Further, the entire approach is based on international accepted standards, means that not only the services have to follow standardized specifications, but also the simulation framework. This allows an even higher level of interoperability: Additional to the service chaining and processing framework described above, an interoperability approach on the simulation side is provided that allows easy integration of new components into existing models without loosing important functionalities like time management and deadlock prevention. The High Level Architecture was chosen as the basic simulation framework. HLA federation approach One method for integration of distributed simulation models is addressed by the High Level Architecture. HLA was developed under the US Department of Defense initially (DOD 1998), and standardized by the IEEE as IEEE 1516. HLA was began to be used in the civil domain at the end of the nineties (Klein and Straßburger 1997; Schulze et al. 1999; Straßburger 2001). It is the only completed standard in this area, and represents the state-of-the-art framework in the distributed simulation domain. A distributed HLA based simulation is called a federation where the distinct members are encapsulated as federates. A federation can be seen as a set of interacting simulators, real players, measuring instruments, etc., that are called federates. The specification is concerned with the interfaces and does not prescribe the internal structure of the federates. All federates communicate via the RTI (runtime infrastructure), which is essentially a qualified communication bus. No direct inter-federate communication is allowed. The communication is performed by separate “ambassadors” for input and output: the RTIAmbassador forwarding calls from the federate to the RTI and the FederateAmbassador which is used as a callback interface for the RTI. Figure 7 illustrates the ambassador paradigm. Figure 7: High Level Architecture Ambassador Paradigm The federation possesses a federation object model (FOM) which describes the object and interaction classes that are utilizable for all federates within the federation. Starting around 2001, there have been a number of experiments in using web services in combination with the High Level Architecture. One interesting approach was presented by Morse et al. (Morse et al. 2003). A prototype HLA federation was built using XMSF compliant web services for communication between individual federation components. XMSF is a kind of a listing of standards and mostly web-based techniques with the intention to provide a common framework for web based modeling and simulation. For the prototype, the RTI calls were formatted using SOAP (IETF 1999; W3C 2003) employing the BEEP (IETF 2002) communication layer to enable bi-directional calls. The approach 1. allowed multiple federate web services to run on the same machine, which overcomes http’s shortcoming of requiring unidirectional service initiation, and 2. enabled composability of repository-based simulations due to its integration into Intraand Internet through firewalls without costly reconfiguration work. The shortcoming of this approach lies in other shortcomings of the HLA specifications. In particular, the absence of management functionalities that permit a controlled start and termination of federates and federations by external processes, which is a result of simulator status and behavior not being specified before joining or after resigning a federation. This gap in the HLA interface specifications prevents Web services (which are based in a more loosely-coupled approach with real-time binding) from completely controlling simulators. Basis management functionalities are only usable after a simulator exists as a physical process and has successfully joined a federation. Achieving distributed, HLA-based simulations hosted on the World Wide Web’s means being able to control them and make use of the simulations results produced. The concept developed by Wytzisk and Simonis (Wytzisk et al. 2003) allows standardized • external initiation of federations, • controlled start up and termination of federates by external management processes, • interactions with running federates, • access to simulations results by external, non-HLA-processes. The DALI prototype The DALI prototype that was developed to demonstrate the use of interoperable services in combination with a High Level Architecture based concept described above. It is used to serve as the platform for additional federates and spatio-temporal services. A case study region in Germany has been chosen that allows access to a (nearly) real-time precipitation sensor network. The simulation model calculates the actual surface runoff for the area of interest. The model (shown in Figure 8) is based on the so-called SHEmodel and is used by a specialized federate called hydrology federate. Figure 8: Hydrological Model (SHE-Model) The entire model makes intensive use of different OGC Web Services that are supported from a High Level Architecture based simulation. Most of the chaining is performed on HLA side. A more intense use of OGC chaining mechanisms is intended to be used in future. The simulation and service system consists on the following OGC and HLA components (for more detailed description see Simonis et al. 2003): • The Sensor Federate The Sensor Federate provides (nearly) real-time precipitation data of the case study area. It is also capable of providing historic rainfall data based on recorded data stored in a database. It is able to act as a pacemaker for the rest of the federation. • The Hydrology Federate The Hydrology Federate implements the SHE model (see Figure 8) and uses the rainfall and other data provided within the federation. It subscribes to the precipitation data and publishes the forecast which can be configured as different forecast scenarios. A four hour forecast has been used as default forecast scenario. • The Observer Federate The observer federate subscribes to all spatial data that is published within the federation. It forwards all data from the sensor federate and the hydrology federate into a database. • The Notification Federate The notification federate is used to inform the user about data that is published into the federation. It could be used to send notifications – using the Web Notification Service (Simonis and Wytzisk 2003) – for any new published data set or it can act as a condition-alert system, sending notification in case that the current measurement falls below or exceeds predefined threshold values. • Sensor Planning Service The SPS is used to parameterize and start the simulation process. The simulation model needs information about numerous parameters, from addresses of the precipitation data providing SCS and the Web Notification Service to the current vegetation period. • Sensor Collection Service Two sensor collection services currently take part in the spatial data infrastructure. The first SCS is accessed from the specialized sensor federate. It provides precipitation measurements for a number of measurement stations. The second SCS is used to access the calculated runoff data resulting from the simulation model (McCarty 2002). • Web Notification Service The Web Notification Service is used to inform the user about new simulated runoff data or in case that any kind of error occurred. • Web Map Service The WMS is used to get a graphical representation of the runoff data. As shown in Figure 9, it produces a map of the area of interest. The runoff forecast in the upper right corner is produces by the web map client but is based on data actually accessed from the Sensor Collection Service. Figure 9: Web Map Service and Sensor Collection Service Client Conclusions The integration of simulated data – dynamically produced by distributed simulation engines – into Spatial Data Infrastructures provides the basis for a much higher level of technical usability of the infrastructures. The two approaches show how a framework of interoperable web services, complemented by a mechanism to exchange data in a standardized way enables a so far unequalled level of cooperation between different data providers and processors. Acknowledgements The figures showing the geological simulation system were generated through the project Targeting new mineralisation in western Victoria running through the Cooperative Research Centre for Predictive Mineral Exploration and were kindly made available by Peter Schaubs of CSIRO, Tim Rawling and Chris Wilson from the University of Melbourne and Jon Dugdale from MPI Mines. The research work of the DALI project was made possible thanks to funding of the Ministry of Science and Research of the State of Northrhine-Westfalia, Germany.